{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### News"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Placeholder"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Installation"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Placeholder"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZkH_y8UC9lvv"
      },
      "source": [
        "### Unsloth"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hzPgFeIkZn9q"
      },
      "source": [
        "# Goal: Make faster kernels with Reinforcement Learning\n",
        "\n",
        "Our goal is to make a faster matrix multiplication kernel by doing RL on GTP-OSS 20B with Unsloth.\n",
        "\n",
        "<img src=\"https://upload.wikimedia.org/wikipedia/commons/thumb/1/18/Matrix_multiplication_qtl1.svg/500px-Matrix_multiplication_qtl1.svg.png\" height=200 />\n",
        "\n",
        "You will learn how to:\n",
        "1. Counteract **reward hacking** like cheating, caching, laziness.\n",
        "2. Timing and correctness of kernels and time limits.\n",
        "3. Making good **reward functions**\n",
        "4. How to seriously do RL to make optimized CUDA kernels"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 575,
          "referenced_widgets": [
            "ce40e56d7a524059a366371d52a5d7f5",
            "99e4796e59e0472eac83077cff768bf0",
            "bb788875516e4aba9ec87406be52eeff",
            "13d9d625e8734afdb9b1d896b2b0992e",
            "154717a8ed9745048a8906af3b221d41",
            "6dda7866d8c14e88be4fbd797417f556",
            "c535608487724a408d60b5d1c54b18b8",
            "856ded5e0d6e491eb38d316f2f43fbd7",
            "d09aa36b14ff466d96b1236be27919de",
            "a0e82f0030454bafb6bfb6bb1c9b3f7b",
            "ea437a5e5ee741ac8ac528362dc45bb6",
            "f0284fe69116476685139bc8ad7d80d2",
            "7dceab848c9043bfbad62c0ac6d9f13c",
            "e25e6d26258a4d5e864a45b7a3107c4f",
            "9190b3de6fcd4e0ca50c20c055eef834",
            "a7b21195b2824db89f19f8695fcb56cd",
            "127fc2d8cfc54b978b2a49f8dc4c1d32",
            "49e3ddd3f92146baa5a8cb017a036252",
            "c7c83df93e1444baae6168970d498ba9",
            "8d2807bfe85447bdad67987460e99168",
            "dea001037d0b479585264f675754415a",
            "e913fb6b7b86498d847c64196478a628",
            "5cae2f7900ca4509a68d345e26c738e5",
            "bbc57ccb80df4cb68c271c3ffcb85bce",
            "9809a8d0fa1d4bf8b5a96996ad44e9c5",
            "a2ee831695754546845b552ab4c880ad",
            "59259ca023ec4f739f723dccb312bca3",
            "df7811d1db294affbb1f7400b6d67393",
            "bfde109172ab414092bd3c0618e6ac77",
            "39fc8be5ba8141d09eed02162d2512bf",
            "380159861a234f5aa9a0737bd7f03bcb",
            "5323ebbabec14d7dad77fc47538eba93",
            "6f3bfc7e482147999b1fcf399e669452",
            "882a51f080ed407ebe01a05b970e48a5",
            "aa9017bafbe84475a843d4116e2d8cc1",
            "9b62f5baa62f4bccb8c2b7da62c280c7",
            "229f715276c941479a98f632b6a4028e",
            "99b4fa0ecb204d41a4d9237b604fb8e5",
            "59fa8c137a1649559e4d3c7486cbdf30",
            "c39b4f53a4fb409fb414b26a5f776eb8",
            "95461ebf2f9d44c9b745da5c35d3ecea",
            "a7be22b2a2cb4d7ebfa5016cd081af86",
            "b0963a510564455ba9182e877470d219",
            "023cb6271415454489dc5999bbba8a1b",
            "6b2ef4d303ee437cb19a2ad0800e3d9e",
            "81aa522133724d51adc0e1f19cc0bf8f",
            "8c2b98be4b2a466d834d2c8b918946f7",
            "e4275bea01d24aad92bf3b52c0f6032f",
            "3314a4aba3e44ab59c7265d2fbb8f101",
            "c7551368a9e74325986547cf00feb51f",
            "6f5b6b3036c34a248657e83d2db979ff",
            "d4a2d9865d6641b4891de7d61d465f12",
            "e299af20255f4b96bb66c649e12dafc8",
            "1b9a8cb9d8644923b2ee8b5b99de26da",
            "7321c679967f4e20b1feb305cb9d5e46",
            "adfde043c5694bc381fef6d0253ee113",
            "89ae701ad8ae4531bccbfaf2df88ee0f",
            "667c64b9f6294f5a85e6653c220d3c19",
            "3011422a32664f0b99d80456f17848b9",
            "7e7be5dbde274fb1bd2b673fa17e3c6c",
            "fe4ac2a3569c44c5813b447609931a2e",
            "79c650567e584ddc9834ef06ef8e7309",
            "2325e2f0d5d54bc685581d47c4045af2",
            "42c96fe6e04743b994950ce8f774b645",
            "46c0c957337744f19ee57dffa144bb88",
            "479521faaed1410d9b18b6e136fdccae",
            "6126ce31d40c4afbbc4f30ff618c5105",
            "6162fa10939c49feb3cbb5a25bd31f70",
            "508965f1eba9451f8d07cc5ce4cd1e09",
            "1f9374e2838647438922c6f3d16c2b6b",
            "2619d2a3049a49c9ab567d5ace2e6da5",
            "d5841e9d82a44eabb3a10c6990770b26",
            "c9e25b142c284eb585d7fef89e9cfb20",
            "9a76637b3afd4c56be2d99c200c5de8e",
            "e397e262339c437593d7159870e4ddd9",
            "011e3d0bb18541a5a2b0c071afb6eb32",
            "0f1c01032192466fa1aca285847f6391",
            "ff63b135c33544e6b8f327817b04e183",
            "39dd71d4a7d741dba0d95c6964207bf8",
            "a26fca95ca7f484fa3fbd89bf3736fa1",
            "451add29f7d84acfad95b1fa96a5cdd2",
            "a461767972644fdda27f372c6c0d983a",
            "2cff36ba4f974bd495bcc5808bc15f7b",
            "84dd3e02fd8249189c93a017f7e1cc93",
            "ab126a046cdd44e0a047abcb1528113d",
            "06fc35ae1cd245c2b711ae3d05004a35",
            "bcedab6c5dd0423b91eea9734e5b6904",
            "528d3a190af640d9ab7a7d7fe086f31c",
            "719839a5728a46248edcda02252e9561",
            "ddfbeb16103f44a08e3d7bb6bf4da86a",
            "1cb6e34da8ad46b3a0057ab01de94fc1",
            "ff698edf61144077bf61fe03369cdf5b",
            "07edfc41f3f44c9d851e12d984f19703",
            "8f439204fa79466aa2d20d0d2a28d74d",
            "805f7c86609347ed8fe945a38111edf3",
            "e6cb50ddb26844a084ae5a7d4a492eca",
            "515df8cf4bcf4fb0aa14705a195aced2",
            "db16c5bfe0be4728a4cacda8573101c4",
            "a4d88610a9464bbf9be3922d760b0670",
            "0a49a476e7c844bd8048e4e9c056fa7b",
            "5b2ec551f1ca490886287c0230ae7e8f",
            "9ddafad46ede4baebbc1f06928bbf01e",
            "d893a89d9c764fbf98459a301ffaa06a",
            "0fc9cecf55314af69389827852f390f3",
            "7c2a5bfc484043fe8c83d766c84b1827",
            "d481cf7937ea4399adf8153a672db38d",
            "c66e476d9fb046499f5dc878844b9aa8",
            "72469601dc8f485dbe1065124f86ad0b",
            "404ff34787884e879ab8e8c1cb99250f",
            "9fe343b36ef1493b86fc6a788bd916ed",
            "db814fd85f5342539537014ff47a4679",
            "85dac41978864ba78784413758a2e393",
            "0c65a1aa0c7243d196e96bd6ef6ad556",
            "e19c28cd15104cff8bff0bc3a1338f3f",
            "c22e6f19fcd949738d3da38670209bf1",
            "194f8f1987c84e9992e6ae7ab8352874",
            "3bd095f10eb24396aeec9d84629190d3",
            "f2072d86f3824135a8c8d4fd0fe4c7f3",
            "744f0a91396f4df6a3e5e4a32a785801",
            "a715051a1252454880f83726c380857f",
            "925b4741bcf24e52a91070b1c2a133d5",
            "291b76fb815b419fb8d12001ffb53dcc",
            "a9e779db1e7940e89b8599d4d19bf672",
            "3f92e70ba76d4cc59309fe9dd9adbd31",
            "4a6ea77ab6e547d0be2db28c3fc6a0de",
            "9efc5450ab4241f7a4668af9d6af33e8",
            "c103676f50c549d981e719d8baece21a",
            "57112275643b4fce8f681ac9d21bc2b1",
            "8b68fbede6a04c95a8996bf9e7945a68",
            "1947120737b344e996a45f935a18c2f0",
            "3e89a1c5e2ed49b280947c51e1713483",
            "7b808206dd32488c8d62888934c3ee12"
          ]
        },
        "id": "DkIvEkIIkEyB",
        "outputId": "3813293f-2f73-452c-8f1f-dc8170eb0e53"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.\n",
            "🦥 Unsloth Zoo will now patch everything to make training faster!\n",
            "==((====))==  Unsloth 2025.9.9: Fast Gpt_Oss patching. Transformers: 4.56.2.\n",
            "   \\\\   /|    Tesla T4. Num GPUs = 1. Max memory: 14.741 GB. Platform: Linux.\n",
            "O^O/ \\_/ \\    Torch: 2.10.0.dev20250927+cu128. CUDA: 7.5. CUDA Toolkit: 12.8. Triton: 3.5.0\n",
            "\\        /    Bfloat16 = FALSE. FA [Xformers = None. FA2 = False]\n",
            " \"-____-\"     Free license: http://github.com/unslothai/unsloth\n",
            "Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!\n",
            "Unsloth: Using float16 precision for gpt_oss won't work! Using float32.\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "ce40e56d7a524059a366371d52a5d7f5",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "model.safetensors.index.json: 0.00B [00:00, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "f0284fe69116476685139bc8ad7d80d2",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Fetching 4 files:   0%|          | 0/4 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "5cae2f7900ca4509a68d345e26c738e5",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "model-00002-of-00004.safetensors:   0%|          | 0.00/4.00G [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "882a51f080ed407ebe01a05b970e48a5",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "model-00003-of-00004.safetensors:   0%|          | 0.00/3.37G [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "6b2ef4d303ee437cb19a2ad0800e3d9e",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "model-00001-of-00004.safetensors:   0%|          | 0.00/4.00G [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "adfde043c5694bc381fef6d0253ee113",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "model-00004-of-00004.safetensors:   0%|          | 0.00/1.16G [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "6126ce31d40c4afbbc4f30ff618c5105",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "ff63b135c33544e6b8f327817b04e183",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "generation_config.json:   0%|          | 0.00/165 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Unsloth: Offloading embeddings to RAM to save 1.08 GB.\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "719839a5728a46248edcda02252e9561",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "tokenizer_config.json: 0.00B [00:00, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "0a49a476e7c844bd8048e4e9c056fa7b",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "tokenizer.json:   0%|          | 0.00/27.9M [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "db814fd85f5342539537014ff47a4679",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "special_tokens_map.json:   0%|          | 0.00/446 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "291b76fb815b419fb8d12001ffb53dcc",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "chat_template.jinja: 0.00B [00:00, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "from unsloth import FastLanguageModel\n",
        "import torch\n",
        "max_seq_length = 768 # Can increase for longer RL output\n",
        "lora_rank = 4 # Larger rank = smarter, but slower\n",
        "model, tokenizer = FastLanguageModel.from_pretrained(\n",
        "    model_name = \"unsloth/gpt-oss-20b\",\n",
        "    max_seq_length = max_seq_length,\n",
        "    load_in_4bit = True, # False for LoRA 16bit\n",
        "    offload_embedding = True, # Reduces VRAM by 1GB\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TfeUs-lQJDSq"
      },
      "source": [
        "We now add some small amount of LoRA weights to GPT-OSS so we only need to train those, instead of training on the full model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8rGa-o3HJCo1",
        "outputId": "3e5a1359-a1aa-4704-e164-c5640d232698"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Unsloth: Making `model.base_model.model.model` require gradients\n"
          ]
        }
      ],
      "source": [
        "model = FastLanguageModel.get_peft_model(\n",
        "    model,\n",
        "    r = lora_rank, # Choose any number > 0 ! Suggested 8, 16, 32, 64, 128\n",
        "    target_modules = [\n",
        "        \"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n",
        "        \"gate_proj\", \"up_proj\", \"down_proj\",\n",
        "    ],\n",
        "    lora_alpha = lora_rank*2, # *2 speeds up training\n",
        "    use_gradient_checkpointing = \"unsloth\", # Reduces memory usage\n",
        "    random_state = 3407,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N0QnO9_YJBOI"
      },
      "source": [
        "# Optimized matrix multiplication\n",
        "\n",
        "Numpy has optimized matrix multiplication kernels for CPUs via BLAS optimized operations. For GPUs, one can use CUDA accelerated cuBLAS kernels which PyTorch calls under the hood.\n",
        "\n",
        "To generate some random matrices to do matrix multiplication, we can do the below:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "D9CI4jtgL5mw"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "def generate_random_matrices(seed = 3407, n = 256):\n",
        "    random_state = np.random.RandomState(seed)\n",
        "    n, k, m = random_state.randint(1, n+1, size = 3)\n",
        "    A = np.random.uniform(-10, 10, size = (n, k))\n",
        "    B = np.random.uniform(-10, 10, size = (k, m))\n",
        "    return A, A.tolist(), B, B.tolist()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4BcaLniVKLpa"
      },
      "source": [
        "We shall generate a small matrix, and see the matrix multiplied output"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-M8kGaFRJ2ic",
        "outputId": "3c5c0ee1-ae76-41d2-d087-8441b6c9177e"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[[-2.8313286   4.54613909 -7.95265309  6.53459836  2.87235103]\n",
            " [ 7.0739631   3.76278879  9.31565599 -8.52884711  9.96832952]\n",
            " [ 8.41214082  6.51136046 -3.79347975 -2.46773693 -2.32292989]\n",
            " [ 3.91302932  4.98335304 -5.33855089  5.71057634 -2.79871647]]\n",
            "[[ 0.39218774 -9.6181377  -3.49736707]\n",
            " [-0.33354865 -1.05626139  3.87231208]\n",
            " [ 0.49494174  5.91863954 -6.83183693]\n",
            " [ 5.1465162  -7.51648113  1.00445384]\n",
            " [ 9.63213377 -4.92327556  3.323014  ]]\n",
            "[[  54.73441488  -87.89725072   97.94605887]\n",
            " [  58.25238906   -1.8467447   -49.25453031]\n",
            " [ -35.82528794  -80.25394462   11.51225408]\n",
            " [  -0.33785799 -103.64132345   38.51974367]]\n"
          ]
        }
      ],
      "source": [
        "A, A_list, B, B_list = generate_random_matrices(seed = 42, n = 5)\n",
        "print(A)\n",
        "print(B)\n",
        "print(np.matmul(A, B))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "envzrXmjKRff"
      },
      "source": [
        "We can call a LLM to generate a simple matrix multiply kernel in Python only, and we can calculate the differences between the actual result and the kernel's result"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "b-gSgthFI_wq"
      },
      "outputs": [],
      "source": [
        "def calculate_difference(pred, real):\n",
        "    if pred is None: return 5, 5\n",
        "    assert real is not None\n",
        "    import numpy as np\n",
        "    try:\n",
        "        difference = pred - real\n",
        "    except:\n",
        "        return 5, 5\n",
        "    amax_error = float(np.amax(difference))\n",
        "    mse_error  = float(np.mean(np.square(difference)))\n",
        "    return amax_error, mse_error"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "q9gmkbTnKbcF"
      },
      "outputs": [],
      "source": [
        "# Kernel generated by GPT-5\n",
        "def matmul(A, B):\n",
        "    z, s = zip, sum\n",
        "    Bt = list(z(*B))\n",
        "    return [[s(a*b for a, b in z(row, col)) for col in Bt] for row in A]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J-WfRwQeKtEZ"
      },
      "source": [
        "We see the error below is very small, so that's good!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_QvIidsPKg2C",
        "outputId": "6aee038b-0c1d-4825-e72e-4cbb28c45e18"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "(7.105427357601002e-15, 4.6783406255758477e-29)"
            ]
          },
          "execution_count": 8,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "prediction = matmul(A_list, B_list)\n",
        "calculate_difference(prediction, np.matmul(A, B))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VR6czU96cpxf"
      },
      "source": [
        "# Countering Reward Hacking\n",
        "\n",
        "The ultimate goal of RL is to maximize some reward (say speed, revenue, some metric).\n",
        "\n",
        "But RL can **cheat** When the RL algorithm learns a trick or exploits something to increase the reward, without actually doing the task at end, this is called \"Reward Hacking\".\n",
        "\n",
        "Some good examples are in https://en.wikipedia.org/wiki/Reward_hacking\n",
        "\n",
        "For matrix multiplication kernels, we might see the following issues:\n",
        "\n",
        "* Laziness: RL learns to use Numpy, Torch, other libraries, which calls optimized CUDA kernels.\n",
        "* Caching: RL learns to cache the result of the output\n",
        "* Cheating: RL learns to find the actual output by inspecting Python global variables\n",
        "* RL learns to edit the timing function to make it output 0 time as passed.\n",
        "\n",
        "And possibly more. We shall try to address each!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tRhLV_bZMYxy"
      },
      "source": [
        "# Countering Reward Hacking 1: Stop laziness\n",
        "We can stop the RL algorithm from calling optimized code by inspecting if the generated code imports other non standard Python libraries. We used GPT-5 to help generate this check `check_only_stdlib_imports`:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "MXPZ1MsUMqNZ"
      },
      "outputs": [],
      "source": [
        "#@title (Collapsible code)\n",
        "import ast\n",
        "import sys\n",
        "import sysconfig\n",
        "from pathlib import Path\n",
        "\n",
        "def _stdlib_names():\n",
        "    \"\"\"\n",
        "    Build a set of canonical stdlib top-level module/package names.\n",
        "    Uses sys.stdlib_module_names when available (3.10+), with a\n",
        "    filesystem fallback for older versions/edge cases.\n",
        "    \"\"\"\n",
        "    names = {m.lower() for m in getattr(sys, \"stdlib_module_names\", set())}\n",
        "    names |= {m.lower() for m in sys.builtin_module_names}\n",
        "    names.add(\"__future__\")  # special-case\n",
        "\n",
        "    # Fallback/augmentation: scan the stdlib directory\n",
        "    try:\n",
        "        stdlib_dir = Path(sysconfig.get_path(\"stdlib\"))\n",
        "        if stdlib_dir.exists():\n",
        "            for p in stdlib_dir.iterdir():\n",
        "                if p.name == \"site-packages\":\n",
        "                    continue\n",
        "                if p.suffix == \".py\":\n",
        "                    names.add(p.stem.lower())\n",
        "                elif p.is_dir() and (p / \"__init__.py\").exists():\n",
        "                    names.add(p.name.lower())\n",
        "    except Exception:\n",
        "        # conservative fallback; the names set above will still work well\n",
        "        pass\n",
        "\n",
        "    return names\n",
        "\n",
        "_STDLIB_SET = _stdlib_names()\n",
        "\n",
        "def check_only_stdlib_imports(code: str):\n",
        "    \"\"\"\n",
        "    Return (ok: bool, details: dict)\n",
        "\n",
        "    ok == True  -> all absolute imports are from the stdlib.\n",
        "    ok == False -> details['non_stdlib'] lists offending top-level modules.\n",
        "\n",
        "    details includes:\n",
        "      - stdlib: sorted list of stdlib imports found\n",
        "      - non_stdlib: sorted list of non-stdlib imports found\n",
        "      - relative_imports: count of relative imports (always allowed here)\n",
        "    \"\"\"\n",
        "    try:\n",
        "        tree = ast.parse(code)\n",
        "    except SyntaxError as e:\n",
        "        return False, {\n",
        "            \"error\": f\"SyntaxError: {e}\",\n",
        "            \"stdlib\": [],\n",
        "            \"non_stdlib\": [],\n",
        "            \"relative_imports\": 0,\n",
        "        }\n",
        "\n",
        "    abs_imports = set()\n",
        "    relative_count = 0\n",
        "\n",
        "    class Visitor(ast.NodeVisitor):\n",
        "        def visit_Import(self, node: ast.Import):\n",
        "            for alias in node.names:\n",
        "                abs_imports.add(alias.name.split(\".\")[0])\n",
        "        def visit_ImportFrom(self, node: ast.ImportFrom):\n",
        "            nonlocal relative_count\n",
        "            if (node.level or 0) > 0:\n",
        "                # relative import\n",
        "                relative_count += 1\n",
        "            else:\n",
        "                if node.module:\n",
        "                    abs_imports.add(node.module.split(\".\")[0])\n",
        "\n",
        "    Visitor().visit(tree)\n",
        "\n",
        "    stdlib_found = sorted(m for m in abs_imports if m.lower() in _STDLIB_SET)\n",
        "    non_stdlib = sorted(m for m in abs_imports if m.lower() not in _STDLIB_SET)\n",
        "\n",
        "    return len(non_stdlib) == 0, {\n",
        "        \"stdlib\": stdlib_found,\n",
        "        \"non_stdlib\": non_stdlib,\n",
        "        \"relative_imports\": relative_count,\n",
        "    }"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ngUAw1lMM9JQ"
      },
      "source": [
        "For example, let's call `check_only_stdlib_imports` on a random piece of matrix multiplication code generated by GPT-5:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zz80kvg6M4BG",
        "outputId": "273b3ddd-540b-4f21-b671-69af0e8cf5f1"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Only stdlib imports? False\n",
            "{'stdlib': [], 'non_stdlib': ['numpy', 'torch'], 'relative_imports': 0}\n"
          ]
        }
      ],
      "source": [
        "sample = \"\"\"\n",
        "def matmul(A, B):\n",
        "    import numpy as np\n",
        "    from torch import matmul\n",
        "    z, s = zip, sum\n",
        "    Bt = list(z(*B))\n",
        "    return [[s(a*b for a, b in z(row, col)) for col in Bt] for row in A]\n",
        "\"\"\"\n",
        "ok, info = check_only_stdlib_imports(sample)\n",
        "print(\"Only stdlib imports?\", ok)\n",
        "print(info)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J6lgkGkEN7B0"
      },
      "source": [
        "# Countering Reward Hacking 2: Stop cheating\n",
        "We can stop the RL algorithm from using global or cached variables by restricting it's `locals` and `globals`.\n",
        "\n",
        "We are also going to use `exec` to create the function, so we have to save the output to an empty dict.\n",
        "\n",
        "We also disallow global variable access."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jrIeYu-lOLSv",
        "outputId": "421497b7-c256-4d09-eb16-984c0e1095d9"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<function matmul(A, B)>"
            ]
          },
          "execution_count": 11,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "output_function = {}\n",
        "exec(sample, {}, output_function)\n",
        "output_function[\"matmul\"]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SDSrjOTLVyQm"
      },
      "source": [
        "We also disallow global variable access via `types.FunctionType(f.__code__, {})`\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GcmYAmohVqw2",
        "outputId": "47b134cc-2733-4cc9-81ab-a6592be9bb0a"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Success\n",
            "name 'np' is not defined\n"
          ]
        }
      ],
      "source": [
        "import types\n",
        "output_function[\"matmul\"] = types.FunctionType(output_function[\"matmul\"].__code__, {})\n",
        "\n",
        "def import_numpy():\n",
        "    np.matmul\n",
        "    print(\"Success\")\n",
        "\n",
        "import_numpy()\n",
        "import_numpy = types.FunctionType(import_numpy.__code__, {})\n",
        "try:\n",
        "    import_numpy()\n",
        "except Exception as e:\n",
        "    print(str(e))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5tJKwLUgZsRq"
      },
      "outputs": [],
      "source": [
        "def create_locked_down_function(function):\n",
        "    output_function = {}\n",
        "    exec(function, {}, output_function)\n",
        "    new_matmul = output_function[\"matmul\"]\n",
        "    new_matmul = types.FunctionType(new_matmul.__code__, {})\n",
        "    return new_matmul"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Sl-IxTZ6Nvm9"
      },
      "source": [
        "# Countering Reward Hacking 3: Stop caching\n",
        "We can stop the RL algorithm from using cached data by wiping the cache with a large fake matrix. We also have to benchmark carefully with multiple loops and turns.\n",
        "\n",
        "We also add a **timer** to not make the algorithm go in an endless loop."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jZwGdNyMNlEV"
      },
      "outputs": [],
      "source": [
        "import os, gc, time, statistics\n",
        "import signal\n",
        "from contextlib import contextmanager\n",
        "class TimeoutError(Exception): pass\n",
        "\n",
        "@contextmanager\n",
        "def time_limit(seconds):\n",
        "    def _handler(signum, frame):\n",
        "        raise TimeoutError(f\"Timed out after {seconds}s\")\n",
        "    old = signal.signal(signal.SIGALRM, _handler)\n",
        "    signal.setitimer(signal.ITIMER_REAL, seconds)\n",
        "    try:\n",
        "        yield\n",
        "    finally:\n",
        "        signal.setitimer(signal.ITIMER_REAL, 0.0)\n",
        "        signal.signal(signal.SIGALRM, old)\n",
        "\n",
        "class Benchmarker:\n",
        "    def __init__(self, trials = 3, loops = 1, timeout = 30):\n",
        "        self.buffer = np.zeros(2 * 1024 * 1024 * 1024, dtype = np.uint8)\n",
        "        self.trials = trials\n",
        "        self.loops = loops\n",
        "        assert timeout > 0 # Cannot be 0 since it won't work!\n",
        "        self.timeout = timeout\n",
        "    def thrash(self):\n",
        "        # Edit the buffer to wipe cache lines\n",
        "        self.buffer ^= 1\n",
        "        return int(self.buffer[::4096].sum())\n",
        "\n",
        "    def benchmark(self, function, arguments):\n",
        "        assert len(arguments) == self.loops\n",
        "        samples = []\n",
        "        exceptions = []\n",
        "        timed_out = 0\n",
        "        for _ in range(self.trials):\n",
        "            gc.collect(); gc.disable(); self.thrash()\n",
        "            t_start = time.perf_counter_ns()\n",
        "            for i in range(self.loops):\n",
        "                try:\n",
        "                    with time_limit(self.timeout):\n",
        "                        function(*arguments[i])\n",
        "                except TimeoutError as e:\n",
        "                    timed_out += 1\n",
        "                except Exception as e:\n",
        "                    exceptions.append(str(e))\n",
        "            t_end = time.perf_counter_ns()\n",
        "            gc.enable()\n",
        "            samples.append((t_end - t_start) // max(1, self.loops))\n",
        "        return {\n",
        "            \"median_ns\": int(statistics.median(samples)),\n",
        "            \"mean_ns\": int(statistics.fmean(samples)),\n",
        "            \"stdev_ns\": int(statistics.pstdev(samples) if len(samples) > 1 else 0),\n",
        "            \"exceptions\" : exceptions,\n",
        "            \"timeouts\" : timed_out,\n",
        "        }"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PV5M0DCyOvon"
      },
      "source": [
        "For example we use our matmul kernel we had, and benchmark it with a 10 second delay:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8df8tZcEOuYJ",
        "outputId": "b4b2cdf0-f35b-485a-e479-c2f082d37939"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'median_ns': 66885199,\n",
              " 'mean_ns': 66885199,\n",
              " 'stdev_ns': 0,\n",
              " 'exceptions': [],\n",
              " 'timeouts': 0}"
            ]
          },
          "execution_count": 15,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "A, A_list, B, B_list = generate_random_matrices(seed = 0, n = 256)\n",
        "Benchmarker(trials = 1, timeout = 10).benchmark(output_function[\"matmul\"], [(A_list, B_list)])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8CzwCyXIPK04"
      },
      "source": [
        "# Data & RL task setup\n",
        "\n",
        "We now have to create a prompt to the model for which it will do some task. For our matrix multiply example, we use the below:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "B-2RRE4HMrQO",
        "outputId": "fc5a4a66-154a-419e-e200-63987fff4204"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Create a new fast matrix multiplication function using only native Python code.\n",
            "You are given a list of list of numbers.\n",
            "Output your new function in backticks using the format below:\n",
            "```python\n",
            "def matmul(A, B):\n",
            "    return ...\n",
            "```\n"
          ]
        }
      ],
      "source": [
        "prompt = \"\"\"\n",
        "Create a new fast matrix multiplication function using only native Python code.\n",
        "You are given a list of list of numbers.\n",
        "Output your new function in backticks using the format below:\n",
        "```python\n",
        "def matmul(A, B):\n",
        "    return ...\n",
        "```\n",
        "\"\"\".strip()\n",
        "print(prompt)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MIdudFUodN4i"
      },
      "source": [
        "First, let's prompt GPT-OSS without RL and see how it goes:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9HJxrS76h3Ds",
        "outputId": "f3616393-c222-47f3-ffc5-9a87bc106e61"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "<|start|>system<|message|>You are ChatGPT, a large language model trained by OpenAI.\n",
            "Knowledge cutoff: 2024-06\n",
            "Current date: 2025-09-29\n",
            "\n",
            "Reasoning: low\n",
            "\n",
            "# Valid channels: analysis, commentary, final. Channel must be included for every message.\n",
            "Calls to these tools must go to the commentary channel: 'functions'.<|end|><|start|>user<|message|>Create a new fast matrix multiplication function using only native Python code.\n",
            "You are given a list of list of numbers.\n",
            "Output your new function in backticks using the format below:\n",
            "```python\n",
            "def matmul(A, B):\n",
            "    return ...\n",
            "```<|end|><|start|>assistant<|channel|>analysis<|message|>We need to provide a quick naive matrix multiplication? \"New fast\" but only native Python. Provide code. Use list comprehension or loops. Optimize maybe by transposing B. Provide code.<|end|><|start|>assistant<|channel|>final<|message|>Here is a simple, fast implementation of matrix multiplication that uses only native Python constructs and avoids unnecessary temporary lists by transposing the second matrix first. It works with any list‑of‑lists representation of a rectangular matrix.\n",
            "\n",
            "```python\n",
            "def matmul(A, B):\n",
            "    \"\"\"\n",
            "    Multiply two matrices A and B where both are given as\n",
            "    lists of lists (rows) and are compatible for multiplication.\n",
            "    Returns the resulting matrix as a list of lists.\n",
            "    \"\"\"\n",
            "    # Transpose B so we can access its columns as rows.\n",
            "    B_T = [list(col) for col in zip(*B)]          # O(n*m) time\n",
            "    result = []\n",
            "\n",
            "    for row in A:                                 # for each row in A\n",
            "        res_row = []\n",
            "        for col in B_T:                           # for each column of B\n",
            "            # Compute dot product of row and column\n",
            "            dot = sum(a * b for a, b in zip(row, col))\n",
            "            res_row.append(dot)\n",
            "        result.append(res_row)\n",
            "\n",
            "    return result\n",
            "```\n",
            "\n",
            "**How it works**\n",
            "\n",
            "1. `zip(*B)` transposes `B` in O(n*m) time, allowing us to traverse columns of `B` as rows, thus improving locality.\n",
            "2. For each row of `A` and column of `B`, we compute the dot product using a generator expression (`sum(a * b for a, b in zip(row, col))`), which keeps the memory footprint small.\n",
            "3. The function returns the resulting matrix as a list of lists.\n",
            "\n",
            "This routine is purely native Python and should be reasonably fast for typical sizes that fit comfortably in memory.<|return|>\n"
          ]
        }
      ],
      "source": [
        "text = tokenizer.apply_chat_template(\n",
        "    [{\"role\": \"user\", \"content\": prompt}],\n",
        "    tokenize = False,\n",
        "    add_generation_prompt = True,\n",
        "    reasoning_effort = \"low\",\n",
        ")\n",
        "\n",
        "from transformers import TextStreamer\n",
        "_ = model.generate(\n",
        "    **tokenizer(text, return_tensors = \"pt\").to(\"cuda\"),\n",
        "    temperature = 1.0,\n",
        "    max_new_tokens = 512,\n",
        "    streamer = TextStreamer(tokenizer, skip_prompt = False),\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iknaWZNudTNq"
      },
      "source": [
        "# Reward functions\n",
        "\n",
        "We now design the `extract_function` function which simply extracts the function wrapped in 3 backticks.\n",
        "\n",
        "And 4 reward functions:\n",
        "\n",
        "1. `function_works` which rewards the model if the strategy is a valid Python function.\n",
        "2. `no_cheating` which checks if the function imported other modules, and if it did, we penalize it.\n",
        "3. `correctness_check` which checks if the kernel was correct or wrong - it shouldn't generate gibberish!\n",
        "4. `speed_check` checks the performance relative to Numpy matmul directly."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8JJGXKdJ-Zl_",
        "outputId": "3d16b32b-990b-4472-c3a3-267c7d6cf47c"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "def matmul(A, B):\n",
            "    return ...\n"
          ]
        }
      ],
      "source": [
        "def extract_function(text):\n",
        "    if text.count(\"```\") >= 2:\n",
        "        first = text.find(\"```\") + 3\n",
        "        second = text.find(\"```\", first)\n",
        "        fx = text[first : second].strip()\n",
        "        fx = fx.removeprefix(\"python\\n\")\n",
        "        fx = fx[fx.find(\"def\"):]\n",
        "        if fx.startswith(\"def matmul(A, B):\"): return fx\n",
        "    return None\n",
        "print(extract_function(prompt))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KLXEcf_HSJlI"
      },
      "source": [
        "Below is our `function_works` reward function which uses Python's `exec` but guarded by not allowing leakage of local and global variables. We can also use `check_only_stdlib_imports` first to check if there are errors before even executing the function:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "h3-B0IIsS56S",
        "outputId": "67486b3d-aeb5-41d8-f4b5-b6ba9cdc97ad"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "(False,\n",
              " {'error': \"SyntaxError: expected '(' (<unknown>, line 1)\",\n",
              "  'stdlib': [],\n",
              "  'non_stdlib': [],\n",
              "  'relative_imports': 0})"
            ]
          },
          "execution_count": 19,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "ok, info = check_only_stdlib_imports(\"def a\")\n",
        "ok, info"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "qgFNXORy-lpO"
      },
      "outputs": [],
      "source": [
        "def function_works(completions, **kwargs):\n",
        "    scores = []\n",
        "    for completion in completions:\n",
        "        score = 0\n",
        "        response = completion[0][\"content\"]\n",
        "        function = extract_function(response)\n",
        "        print(function)\n",
        "        if function is not None:\n",
        "            ok, info = check_only_stdlib_imports(function)\n",
        "        if function is None or \"error\" in info:\n",
        "            score = -2.0\n",
        "        else:\n",
        "            try:\n",
        "                new_matmul = create_locked_down_function(function)\n",
        "                score = 1.0\n",
        "            except:\n",
        "                score = -0.5\n",
        "        scores.append(score)\n",
        "    return scores"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Gf69i2WT-m4K"
      },
      "source": [
        "`no_cheating` checks if the function cheated since it might have imported Numpy or Torch optimized code."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cUfHzCVx-nGK"
      },
      "outputs": [],
      "source": [
        "def no_cheating(completions, **kwargs):\n",
        "    scores = []\n",
        "    for completion in completions:\n",
        "        score = 0\n",
        "        response = completion[0][\"content\"]\n",
        "        function = extract_function(response)\n",
        "        if function is not None:\n",
        "            ok, info = check_only_stdlib_imports(function)\n",
        "        else:\n",
        "            ok = False\n",
        "        scores.append(1.0 if ok else -20.0) # Penalize heavily!\n",
        "    return scores"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "slnqWG3FTror"
      },
      "source": [
        "Next `correctness_check` checks if the kernel was correct. We want to penalize if the absolute error is larger than 1, and if the mean squared error is somewhat bigger then machine epsilon.\n",
        "\n",
        "We have to execute the code now!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cFBp-MkyYeoE",
        "outputId": "5d065a27-553a-4599-bfaa-375a7570357c"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "np.float64(2.220446049250313e-16)"
            ]
          },
          "execution_count": 22,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "np.finfo(np.float64).eps"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "sNi129lYTpZ2"
      },
      "outputs": [],
      "source": [
        "def correctness_check(completions, **kwargs):\n",
        "    scores = []\n",
        "    # Generate some random matrices of size less than 128\n",
        "    A, A_list, B, B_list = generate_random_matrices(seed = np.random.randint(10000), n = 128)\n",
        "    for completion in completions:\n",
        "        score = 0\n",
        "        response = completion[0][\"content\"]\n",
        "        function = extract_function(response)\n",
        "        if function is not None:\n",
        "            ok, info = check_only_stdlib_imports(function)\n",
        "        if function is None or \"error\" in info:\n",
        "            scores.append(0)\n",
        "            continue\n",
        "        try:\n",
        "            new_matmul = create_locked_down_function(function)\n",
        "        except:\n",
        "            scores.append(0)\n",
        "            continue\n",
        "        try:\n",
        "            pred = new_matmul(A_list.copy(), B_list.copy())\n",
        "        except:\n",
        "            # Failed!\n",
        "            scores.append(-2.0)\n",
        "            continue\n",
        "        true = np.matmul(A, B)\n",
        "        amax_error, mse_error = calculate_difference(pred, true)\n",
        "\n",
        "        # Check correctness and score!\n",
        "        machine_epsilon = 100*np.finfo(np.float64).eps\n",
        "        if   amax_error >= 3:   score = -3.0\n",
        "        elif amax_error >= 2:   score = -2.5\n",
        "        elif amax_error >= 1:   score = -2.0\n",
        "        elif amax_error >= 0.5: score = -1.0\n",
        "        elif amax_error >= 100*machine_epsilon: score = 0.0\n",
        "        elif amax_error >= machine_epsilon: score = 1.0\n",
        "        else: score = 3.0\n",
        "\n",
        "        if   mse_error >= 3:   score += -3.0\n",
        "        elif mse_error >= 2:   score += -2.5\n",
        "        elif mse_error >= 1:   score += -2.0\n",
        "        elif mse_error >= 0.5: score += -1.0\n",
        "        elif mse_error >= 100*machine_epsilon: score += 0.0\n",
        "        elif mse_error >= machine_epsilon: score += 1.0\n",
        "        else: score += 3.0\n",
        "        scores.append(score)\n",
        "    return scores"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CpTrofI9ZIn8"
      },
      "source": [
        "Finally our benchmarking function for `speed_check`! We shall limit the timer to 10 seconds and do 3 trials."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "w5xkIAzuZMnO",
        "outputId": "e86c1d87-6eb4-4858-9fe1-1123efb6f5df"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'median_ns': 235488,\n",
              " 'mean_ns': 265081,\n",
              " 'stdev_ns': 46595,\n",
              " 'exceptions': [],\n",
              " 'timeouts': 0}"
            ]
          },
          "execution_count": 24,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "A, A_list, B, B_list = generate_random_matrices(seed = 0, n = 256)\n",
        "benchmarker = Benchmarker(trials = 3, timeout = 10)\n",
        "numpy_results = benchmarker.benchmark(np.matmul, [(A, B)])\n",
        "numpy_results"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mNDc6skFZZW6",
        "outputId": "0a012106-7517-46f9-c995-fe52841cbdd5"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'median_ns': 89670,\n",
              " 'mean_ns': 91382,\n",
              " 'stdev_ns': 3139,\n",
              " 'exceptions': [],\n",
              " 'timeouts': 0}"
            ]
          },
          "execution_count": 25,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "new_matmul = create_locked_down_function(extract_function(prompt))\n",
        "new_results = benchmarker.benchmark(new_matmul, [(A_list, B_list)])\n",
        "new_results"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "noUAaX24aqNS"
      },
      "source": [
        "We can take the difference and do a negative sign for slower ones. If the ratio is less than 1 (ie faster, we shall invert it!)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0IT9nXcjaI-X",
        "outputId": "41ba4700-da7e-45f4-e7dd-195a069bda74"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "0.026261625961860154"
            ]
          },
          "execution_count": 26,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "negative = -(new_results[\"median_ns\"] / numpy_results[\"median_ns\"]) / 100\n",
        "positive = +(numpy_results[\"median_ns\"] / new_results[\"median_ns\"]) / 100\n",
        "reward = negative if new_results[\"median_ns\"] >= numpy_results[\"median_ns\"] else positive\n",
        "reward"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ntYNFV0ra-MX",
        "outputId": "b6b445eb-60be-49c7-edb5-c604fbf03f2b"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "3.333333333333333"
            ]
          },
          "execution_count": 27,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "new_results[\"median_ns\"] = 3\n",
        "numpy_results[\"median_ns\"] = 1000\n",
        "negative = -(new_results[\"median_ns\"] / numpy_results[\"median_ns\"]) / 100\n",
        "positive = +(numpy_results[\"median_ns\"] / new_results[\"median_ns\"]) / 100\n",
        "reward = negative if new_results[\"median_ns\"] >= numpy_results[\"median_ns\"] else positive\n",
        "reward"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "HHmXmAxtbVpP"
      },
      "outputs": [],
      "source": [
        "import gc\n",
        "def speed_check(completions, **kwargs):\n",
        "    scores = []\n",
        "    # Generate some random matrices of size less than 256\n",
        "    A, A_list, B, B_list = generate_random_matrices(seed = np.random.randint(10000), n = 256)\n",
        "    numpy_results = benchmarker.benchmark(np.matmul, [(A, B)])\n",
        "    for completion in completions:\n",
        "        score = 0\n",
        "        response = completion[0][\"content\"]\n",
        "        function = extract_function(response)\n",
        "        if function is not None:\n",
        "            ok, info = check_only_stdlib_imports(function)\n",
        "        if function is None or \"error\" in info:\n",
        "            scores.append(0)\n",
        "            continue\n",
        "        try:\n",
        "            new_matmul = create_locked_down_function(function)\n",
        "        except:\n",
        "            scores.append(0)\n",
        "            continue\n",
        "        new_results = benchmarker.benchmark(new_matmul, [(A_list.copy(), B_list.copy())])\n",
        "\n",
        "        # Get score and clip to -10, 10\n",
        "        negative = -(new_results[\"median_ns\"] / numpy_results[\"median_ns\"]) / 100\n",
        "        positive = +(numpy_results[\"median_ns\"] / new_results[\"median_ns\"]) / 100\n",
        "        score = negative if new_results[\"median_ns\"] >= numpy_results[\"median_ns\"] else positive\n",
        "        if score >= 10:  score = 10\n",
        "        if score <= -10: score = -10\n",
        "        scores.append(score)\n",
        "    # Free memory to counteract OOMs\n",
        "    gc.collect()\n",
        "    torch.cuda.empty_cache()\n",
        "    return scores"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TCpSxtvSeAG_"
      },
      "source": [
        "We create the dataset which includes a replica of our prompt. Remember to add reasoning effort of low!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ldf6SjLHVPRv",
        "outputId": "aa498168-bd8c-4352-ddf1-5b36ecc32b2a"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "49\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "{'prompt': [{'content': 'Create a new fast matrix multiplication function using only native Python code.\\nYou are given a list of list of numbers.\\nOutput your new function in backticks using the format below:\\n```python\\ndef matmul(A, B):\\n    return ...\\n```',\n",
              "   'role': 'user'}],\n",
              " 'answer': 0}"
            ]
          },
          "execution_count": 29,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from datasets import Dataset\n",
        "dataset = Dataset.from_list([{\"prompt\" : [{\"role\": \"user\", \"content\": prompt.strip()}], \"answer\" : 0, \"reasoning_effort\": \"low\"}]*1000)\n",
        "maximum_length = len(tokenizer(prompt.strip())[\"input_ids\"])\n",
        "print(maximum_length)\n",
        "dataset[0]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9-IOMhVg-2AM"
      },
      "source": [
        "<a name=\"Train\"></a>\n",
        "### Train the model\n",
        "\n",
        "Now set up GRPO Trainer and all configurations! We also support GSDP, GAPO, Dr GRPO and more! Go to our docs https://docs.unsloth.ai/ for more info!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ptqkXK2D4d6p",
        "outputId": "6c93c354-d103-4e10-e741-35d052b741ab"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Unsloth: We now expect `per_device_train_batch_size` to be a multiple of `num_generations`.\n",
            "We will change the batch size of 1 to the `num_generations` of 2\n"
          ]
        }
      ],
      "source": [
        "max_prompt_length = maximum_length + 1 # + 1 just in case!\n",
        "max_completion_length = max_seq_length - max_prompt_length\n",
        "\n",
        "from trl import GRPOConfig, GRPOTrainer\n",
        "training_args = GRPOConfig(\n",
        "    temperature = 1.0,\n",
        "    learning_rate = 5e-5,\n",
        "    weight_decay = 0.001,\n",
        "    warmup_ratio = 0.1,\n",
        "    lr_scheduler_type = \"linear\",\n",
        "    optim = \"adamw_8bit\",\n",
        "    logging_steps = 1,\n",
        "    per_device_train_batch_size = 1,\n",
        "    gradient_accumulation_steps = 1, # Increase to 4 for smoother training\n",
        "    num_generations = 2, # Decrease if out of memory\n",
        "    max_prompt_length = max_prompt_length,\n",
        "    max_completion_length = max_completion_length,\n",
        "    # num_train_epochs = 1, # Set to 1 for a full training run\n",
        "    max_steps = 100,\n",
        "    save_steps = 100,\n",
        "    report_to = \"none\", # Can use Weights & Biases\n",
        "    output_dir = \"outputs\",\n",
        "\n",
        "    # For optional training + evaluation\n",
        "    # fp16_full_eval = True,\n",
        "    # per_device_eval_batch_size = 4,\n",
        "    # eval_accumulation_steps = 1,\n",
        "    # eval_strategy = \"steps\",\n",
        "    # eval_steps = 1,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r9Mv8UZO5hz-"
      },
      "source": [
        "And let's run the trainer! If you scroll up, you'll see a table of rewards. The goal is to see the `reward` column increase!\n",
        "\n",
        "You might have to wait 150 to 200 steps for any action. You'll probably get 0 reward for the first 100 steps. Please be patient!\n",
        "\n",
        "| Step | Training Loss | reward    | reward_std | completion_length | kl       |\n",
        "|------|---------------|-----------|------------|-------------------|----------|\n",
        "| 1    | 0.000000      | 0.125000  | 0.000000   | 200.000000        | 0.000000 |\n",
        "| 2    | 0.000000      | 0.072375  | 0.248112   | 200.000000        | 0.000000 |\n",
        "| 3    | 0.000000      | -0.079000 | 0.163776   | 182.500000        | 0.000005 |\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vzOuSVCL_GA9",
        "outputId": "c1f2d72a-0633-4bb8-a462-49ac2de2f25a"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Unsloth: Switching to float32 training since model cannot work with float16\n"
          ]
        }
      ],
      "source": [
        "# For optional training + evaluation\n",
        "# new_dataset = dataset.train_test_split(test_size = 0.01)\n",
        "\n",
        "trainer = GRPOTrainer(\n",
        "    model = model,\n",
        "    processing_class = tokenizer,\n",
        "    reward_funcs = [\n",
        "        function_works,\n",
        "        no_cheating,\n",
        "        correctness_check,\n",
        "        speed_check,\n",
        "    ],\n",
        "    args = training_args,\n",
        "    train_dataset = dataset,\n",
        "\n",
        "    # For optional training + evaluation\n",
        "    # train_dataset = new_dataset[\"train\"],\n",
        "    # eval_dataset = new_dataset[\"test\"],\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fQhtuwP4cf34"
      },
      "source": [
        "And let's train the model!\n",
        "\n",
        "**NOTE** A T4 free GPU might take 5 minutes for one generation sadly since it's an old GPU - A100 or H100 will be much faster!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 32,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "VGRxPdSCcfC3",
        "outputId": "7535481b-b1ac-491d-e0e8-b91b2d93f81d"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': 199998, 'pad_token_id': 200017}.\n",
            "==((====))==  Unsloth - 2x faster free finetuning | Num GPUs used = 2\n",
            "   \\\\   /|    Num examples = 1,000 | Num Epochs = 1 | Total steps = 100\n",
            "O^O/ \\_/ \\    Batch size per device = 2 | Gradient accumulation steps = 1\n",
            "\\        /    Data Parallel GPUs = 1 | Total batch size (2 x 1 x 1) = 2\n",
            " \"-____-\"     Trainable parameters = 1,990,656 of 20,916,747,840 (0.01% trained)\n",
            "`generation_config` default values have been modified to match model-specific defaults: {'max_length': 131072}. If this is not desired, please set these values explicitly.\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "def matmul(A, B):\n",
            "    n=len(A); m=len(B[0]); p=len(B)\n",
            "    res=[[0]*m for _ in range(n)]\n",
            "    for i in range(n):\n",
            "        Ai=A[i]\n",
            "        for k in range(p):\n",
            "            aik=Ai[k]\n",
            "            if aik:\n",
            "                Bk=B[k]\n",
            "                for j in range(m):\n",
            "                    res[i][j] += aik*Bk[j]\n",
            "    return res\n",
            "None\n"
          ]
        },
        {
          "data": {
            "text/html": [
              "\n",
              "    <div>\n",
              "      \n",
              "      <progress value='17' max='100' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [ 17/100 2:18:18 < 12:45:18, 0.00 it/s, Epoch 0.02/1]\n",
              "    </div>\n",
              "    <table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              " <tr style=\"text-align: left;\">\n",
              "      <th>Step</th>\n",
              "      <th>Training Loss</th>\n",
              "      <th>reward</th>\n",
              "      <th>reward_std</th>\n",
              "      <th>completions / mean_length</th>\n",
              "      <th>completions / min_length</th>\n",
              "      <th>completions / max_length</th>\n",
              "      <th>completions / clipped_ratio</th>\n",
              "      <th>completions / mean_terminated_length</th>\n",
              "      <th>completions / min_terminated_length</th>\n",
              "      <th>completions / max_terminated_length</th>\n",
              "      <th>kl</th>\n",
              "      <th>rewards / function_works / mean</th>\n",
              "      <th>rewards / function_works / std</th>\n",
              "      <th>rewards / no_cheating / mean</th>\n",
              "      <th>rewards / no_cheating / std</th>\n",
              "      <th>rewards / correctness_check / mean</th>\n",
              "      <th>rewards / correctness_check / std</th>\n",
              "      <th>rewards / speed_check / mean</th>\n",
              "      <th>rewards / speed_check / std</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>1</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-9.362819</td>\n",
              "      <td>17.871674</td>\n",
              "      <td>707.000000</td>\n",
              "      <td>696.000000</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>696.000000</td>\n",
              "      <td>696.000000</td>\n",
              "      <td>696.000000</td>\n",
              "      <td>0.000105</td>\n",
              "      <td>-0.500000</td>\n",
              "      <td>2.121320</td>\n",
              "      <td>-9.500000</td>\n",
              "      <td>14.849242</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>2.828427</td>\n",
              "      <td>-1.362819</td>\n",
              "      <td>1.927317</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>2</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-10.082956</td>\n",
              "      <td>16.853245</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000107</td>\n",
              "      <td>-0.500000</td>\n",
              "      <td>2.121320</td>\n",
              "      <td>-9.500000</td>\n",
              "      <td>14.849242</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>2.828427</td>\n",
              "      <td>-2.082957</td>\n",
              "      <td>2.945746</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>3</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-10.731101</td>\n",
              "      <td>15.936629</td>\n",
              "      <td>714.500000</td>\n",
              "      <td>711.000000</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>711.000000</td>\n",
              "      <td>711.000000</td>\n",
              "      <td>711.000000</td>\n",
              "      <td>0.000058</td>\n",
              "      <td>-0.500000</td>\n",
              "      <td>2.121320</td>\n",
              "      <td>-9.500000</td>\n",
              "      <td>14.849242</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>2.828427</td>\n",
              "      <td>-2.731101</td>\n",
              "      <td>3.862360</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>4</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-9.106020</td>\n",
              "      <td>18.234840</td>\n",
              "      <td>583.500000</td>\n",
              "      <td>506.000000</td>\n",
              "      <td>661.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>583.500000</td>\n",
              "      <td>506.000000</td>\n",
              "      <td>661.000000</td>\n",
              "      <td>0.000762</td>\n",
              "      <td>-0.500000</td>\n",
              "      <td>2.121320</td>\n",
              "      <td>-9.500000</td>\n",
              "      <td>14.849242</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>2.828427</td>\n",
              "      <td>-1.106019</td>\n",
              "      <td>1.564148</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>5</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-9.679836</td>\n",
              "      <td>17.423344</td>\n",
              "      <td>574.000000</td>\n",
              "      <td>430.000000</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>430.000000</td>\n",
              "      <td>430.000000</td>\n",
              "      <td>430.000000</td>\n",
              "      <td>0.000539</td>\n",
              "      <td>-0.500000</td>\n",
              "      <td>2.121320</td>\n",
              "      <td>-9.500000</td>\n",
              "      <td>14.849242</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>2.828427</td>\n",
              "      <td>-1.679837</td>\n",
              "      <td>2.375648</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>6</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-9.139865</td>\n",
              "      <td>18.186977</td>\n",
              "      <td>717.000000</td>\n",
              "      <td>716.000000</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>716.000000</td>\n",
              "      <td>716.000000</td>\n",
              "      <td>716.000000</td>\n",
              "      <td>0.000072</td>\n",
              "      <td>-0.500000</td>\n",
              "      <td>2.121320</td>\n",
              "      <td>-9.500000</td>\n",
              "      <td>14.849242</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>2.828427</td>\n",
              "      <td>-1.139865</td>\n",
              "      <td>1.612013</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>7</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-22.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000075</td>\n",
              "      <td>-2.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-20.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>8</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-0.399041</td>\n",
              "      <td>5.049094</td>\n",
              "      <td>472.500000</td>\n",
              "      <td>227.000000</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>227.000000</td>\n",
              "      <td>227.000000</td>\n",
              "      <td>227.000000</td>\n",
              "      <td>0.001335</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>7.071068</td>\n",
              "      <td>-1.399041</td>\n",
              "      <td>2.021974</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>9</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-13.000000</td>\n",
              "      <td>12.727922</td>\n",
              "      <td>682.000000</td>\n",
              "      <td>646.000000</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>646.000000</td>\n",
              "      <td>646.000000</td>\n",
              "      <td>646.000000</td>\n",
              "      <td>0.000316</td>\n",
              "      <td>-0.500000</td>\n",
              "      <td>2.121320</td>\n",
              "      <td>-9.500000</td>\n",
              "      <td>14.849242</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>2.828427</td>\n",
              "      <td>-5.000000</td>\n",
              "      <td>7.071068</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>10</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-8.128263</td>\n",
              "      <td>19.617599</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000122</td>\n",
              "      <td>-0.500000</td>\n",
              "      <td>2.121320</td>\n",
              "      <td>-9.500000</td>\n",
              "      <td>14.849242</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>2.828427</td>\n",
              "      <td>-0.128262</td>\n",
              "      <td>0.181390</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>11</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.380087</td>\n",
              "      <td>6.165328</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000194</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>7.071068</td>\n",
              "      <td>-0.619913</td>\n",
              "      <td>0.905740</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>12</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.407544</td>\n",
              "      <td>6.176902</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000341</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>7.071068</td>\n",
              "      <td>-0.592456</td>\n",
              "      <td>0.894165</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>13</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-3.978179</td>\n",
              "      <td>0.000136</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000255</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-6.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.021821</td>\n",
              "      <td>0.000135</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>14</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-10.987790</td>\n",
              "      <td>15.573617</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000231</td>\n",
              "      <td>-0.500000</td>\n",
              "      <td>2.121320</td>\n",
              "      <td>-9.500000</td>\n",
              "      <td>14.849242</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.414214</td>\n",
              "      <td>0.012210</td>\n",
              "      <td>0.017268</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>15</td>\n",
              "      <td>0.000100</td>\n",
              "      <td>-8.839410</td>\n",
              "      <td>18.611885</td>\n",
              "      <td>552.500000</td>\n",
              "      <td>387.000000</td>\n",
              "      <td>718.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>387.000000</td>\n",
              "      <td>387.000000</td>\n",
              "      <td>387.000000</td>\n",
              "      <td>0.146420</td>\n",
              "      <td>-0.500000</td>\n",
              "      <td>2.121320</td>\n",
              "      <td>-9.500000</td>\n",
              "      <td>14.849242</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>2.828427</td>\n",
              "      <td>-0.839410</td>\n",
              "      <td>1.187105</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table><p>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "def matmul(A, B):\n",
            "    # A and B are lists of lists\n",
            "    # assume A has shape (m, n) and B has shape (n, p)\n",
            "    # returns matrix (m,p) as list of lists\n",
            "    m = len(A)\n",
            "    n = len(A[0]) if A else 0\n",
            "    p = len(B[0]) if B else 0\n",
            "    # Initialize result matrix\n",
            "    res = [[0]*p for _ in range(m)]\n",
            "    # iterate over A's rows\n",
            "    for i in range(m):\n",
            "        a_row = A[i]\n",
            "        for k in range(n):\n",
            "            a_val = a_row[k]\n",
            "            if a_val == 0:\n",
            "                continue\n",
            "            # Multiply by column of B\n",
            "            b_col_k = B[k]  # Actually for each column j we need b_col_j? but we can iterate over j.\n",
            "            for j in range(p):\n",
            "                res[i][j] += a_val * B[k][j]\n",
            "    return res\n",
            "def matmul(A, B):\n",
            "    # Validate input: A is list of lists. B too. We assume each row is a list. \n",
            "    # We can compute number of rows in A: m = len(A). K = len(A[0]) if not each row? But we need to confirm each row length is same.\n",
            "    # For safe we can compute standard row length from first row of A and from first row of B.\n",
            "    # There should be at least one row in A and B. We'll enforce same row length for each row.\n",
            "    # The number of columns of B must match number of rows of B.\n",
            "    # Actually shape check: A: m rows by k columns. B: k rows by n columns. \n",
            "    # So we need to validate that len(A[0]) equals len(B) correct. \n",
            "    # If fails, raise. \n",
            "    # Then product: matrix C shape m x n.\n",
            "    # Use loops: For i in range(m):\n",
            "    #    row C[i] = [sum(A[i][p] * B[p][j] for p in range(k)) for j in range(n)]\n",
            "    # Each loop uses simple Python operations to compute the sum. Should be fine.\n",
            "Unsloth: Will smartly offload gradients to save VRAM!\n",
            "def matmul(A, B):\n",
            "    # A: m x n, B: n x p\n",
            "    m, n = len(A), len(A[0]) # error if A empty\n",
            "    # verify shape of B\n",
            "    if len(B) != n: raise ValueError(\"Incompatible dimensions\")\n",
            "    p = len(B[0])\n",
            "    # compute result matrix\n",
            "    result = [[0]*p for _ in range(m)]\n",
            "    for i in range(m):\n",
            "        for k in range(n):\n",
            "            aik = A[i][k]\n",
            "            if aik:\n",
            "                for j in range(p):\n",
            "                    result[i][j] += aik*B[k][j]\n",
            "    return result\n",
            "None\n",
            "def matmul(A, B):\n",
            "    \"\"\"\n",
            "    Multiply two matrices A and B using only native Python code.\n",
            "\n",
            "    Parameters\n",
            "    ----------\n",
            "    A : list[list[float]]\n",
            "        First matrix, where each element is a row (list of numbers).\n",
            "    B : list[list[float]]\n",
            "        Second matrix, in the same format as A.\n",
            "\n",
            "    Returns\n",
            "    -------\n",
            "    list[list[float]]\n",
            "        Result of the matrix product A @ B.\n",
            "\n",
            "    Raises\n",
            "    ------\n",
            "    ValueError\n",
            "        If the matrices cannot be multiplied (mismatch in dimensions).\n",
            "    \"\"\"\n",
            "    # Verify A and B are valid matrices\n",
            "    if not A or not B:\n",
            "        raise ValueError(\"Both matrices must be non-empty.\")\n",
            "\n",
            "    n_rows_a, n_cols_a = len(A), len(A[0])\n",
            "    n_rows_b, n_cols_b = len(B), len(B[0])\n",
            "\n",
            "    if n_cols_a != n_rows_b:\n",
            "        raise ValueError(\"Number of columns in A must equal number of rows in B.\")\n",
            "\n",
            "    # Pre‑allocate the result matrix\n",
            "    result = [[0.0 for _ in range(n_cols_b)] for _ in range(n_rows_a)]\n",
            "\n",
            "    # Standard triple‑nested loop\n",
            "    for i in range(n_rows_a):\n",
            "        for k in range(n_cols_a):\n",
            "            a_ik = A[i][k]\n",
            "            for j in range(n_cols_b):\n",
            "                result[i][j] += a_ik * B[k][j]\n",
            "\n",
            "    return result\n",
            "None\n",
            "None\n",
            "def matmul(A, B):\n",
            "    n = len(A)\n",
            "    m = len(B[0])\n",
            "    result = [[0]*m for _ in range(n)]\n",
            "    # Ensure B is square? Not necessary.\n",
            "    for i in range(n):\n",
            "        # A[i] row\n",
            "        Ai = A[i]\n",
            "        res_row = result[i]\n",
            "        for k, a_val in enumerate(Ai):\n",
            "            if a_val:\n",
            "                Bk = B[k]\n",
            "                for j in range(m):\n",
            "                    res_row[j] += a_val * Bk[j]\n",
            "    return result\n",
            "None\n",
            "def matmul(A, B):\n",
            "    \"\"\"\n",
            "    Multiply two matrices A and B using native Python code.\n",
            "    A: list of lists of numbers, each sublist is a row\n",
            "    B: list of lists of numbers, each sublist is a row\n",
            "\n",
            "    Returns a new matrix as a list of lists.\n",
            "    \"\"\"\n",
            "    m = len(A)           # number of rows in A\n",
            "    n = len(A[0])        # number of columns in A (length of a row)\n",
            "    p = len(B[0])        # number of columns in B\n",
            "\n",
            "    # 1️⃣ Pre‑allocate the result matrix\n",
            "    C = [[0] * p for _ in range(m)]\n",
            "\n",
            "    # 2️⃣ Do the multiplication\n",
            "    for i in range(m):\n",
            "        row = A[i]\n",
            "        for k in range(n):\n",
            "            aik = row[k]\n",
            "            if aik:          # some small speedup if aik == 0\n",
            "                col = B[k]\n",
            "                for j in range(p):\n",
            "                    C[i][j] += aik * col[j]\n",
            "    return C\n",
            "None\n",
            "None\n",
            "def matmul(A, B):\n",
            "    m, n = len(A), len(A[0])\n",
            "    p = len(B[0])\n",
            "    assert len(B) == n, \"Incompatible dimensions.\"\n",
            "    res = [[0] * p for _ in range(m)]\n",
            "    for i in range(m):\n",
            "        for k in range(n):\n",
            "            aik = A[i][k]\n",
            "            for j in range(p):\n",
            "                res[i][j] += aik * B[k][j]\n",
            "    return res\n",
            "def matmul(A, B):\n",
            "    ...\n",
            "None\n",
            "def matmul(A, B):\n",
            "    n = len(A)\n",
            "    m = len(B[0])\n",
            "    p = len(B)\n",
            "    # using list comprehension\n",
            "    return [[sum(A[i][k] * B[k][j] for k in range(p)) for j in range(m)] for i in range(n)]\n",
            "def matmul(A, B):\n",
            "    m = len(A)\n",
            "    n = len(B[0]) if B else 0\n",
            "    p = len(B)  # number of rows of B\n",
            "    # Build columns of B for fast access\n",
            "    colB = [ [B[i][j] for i in range(p)] for j in range(n) ]\n",
            "    result = [[0]*n for _ in range(m)]\n",
            "    for i in range(m):\n",
            "        rowA = A[i]\n",
            "        res_row = result[i]\n",
            "        for j in range(n):\n",
            "            acc = 0\n",
            "            col = colB[j]\n",
            "            for k in range(p):\n",
            "                acc += rowA[k] * col[k]\n",
            "            res_row[j] = acc\n",
            "    return result\n",
            "def matmul(A, B):\n",
            "    # Assuming A is m x p, B is p x n\n",
            "    # produce result matrix A * B\n",
            "def matmul(A, B): return ...\n",
            "def matmul(A, B):\n",
            "    return [[sum(A[i][k] * B[k][j] for k in range(len(B))) for j in range(len(B[0]))] for i in range(len(A))]\n",
            "def matmul(A, B):\n",
            "    return ...\n",
            "def matmul(A, B):\n",
            "    if not A or not B:\n",
            "        return []\n",
            "    m, n = len(A), len(A[0])\n",
            "    p = len(B[0])\n",
            "    # optional shape check\n",
            "    # compute result matrix\n",
            "    B_columns = [list(col) for col in zip(*B)]\n",
            "    result = []\n",
            "    for i in range(m):\n",
            "        row = []\n",
            "        Ai = A[i]\n",
            "        for j in range(p):\n",
            "            s = 0\n",
            "            Bj = B_columns[j]\n",
            "            for k in range(n):\n",
            "                s += Ai[k] * Bj[k]\n",
            "            row.append(s)\n",
            "        result.append(row)\n",
            "    return result\n",
            "def matmul(A, B):\n",
            "    ...\n",
            "def matmul(A, B):\n",
            "    ...\n",
            "def matmul(A, B):\n",
            "    # check\n",
            "    ...\n",
            "    # 1x1 case\n",
            "    # recursion for odd size\n",
            "    ...\n",
            "    return result\n",
            "None\n",
            "def matmul(A, B):\n",
            "    # Determine dimensions\n",
            "    n = len(A)\n",
            "    m = len(A[0])  # number of columns in A\n",
            "    p = len(B[0])  # number of columns in B\n",
            "\n",
            "    # Fill the result matrix with zeros\n",
            "    result = [[0 for _ in range(p)] for __ in range(n)]\n",
            "\n",
            "    # Perform multiplication : matrix product\n",
            "    for i in range(n):\n",
            "        for j in range(p):\n",
            "            sum = 0\n",
            "            for k in range(m):   # m == len(B) automatically\n",
            "                sum += A[i][k] * B[k][j]\n",
            "            result[i][j] = sum\n",
            "    return result\n",
            "None\n",
            "def matmul(A, B):\n",
            "    # A and B are lists of lists (rows) with compatible dimensions\n",
            "    m, n, p = len(A), len(A[0]), len(B[0])\n",
            "    # Initialize result matrix with zeros\n",
            "    C = [[0.0] * p for _ in range(m)]\n",
            "    # Multiply\n",
            "    for i in range(m):\n",
            "        a_row = A[i]\n",
            "        c_row = C[i]\n",
            "        for k in range(n):\n",
            "            aik = a_row[k]\n",
            "            if aik:  # skip zero entries\n",
            "                b_col_offset = 0\n",
            "                for j in range(p):\n",
            "                    c_row[j] += aik * B[k][j]\n",
            "    return C\n",
            "def matmul(A, B):\n",
            "    # A: m x n matrix\n",
            "    # B: n x p matrix\n",
            "    # return m x p matrix\n",
            "    m, n = len(A), len(A[0]) if A else 0\n",
            "    p = len(B[0]) if B else 0\n",
            "    # Check sizes\n",
            "    if len(B) != n:\n",
            "        raise ValueError('Incompatible matrix dimensions')\n",
            "    result = [[0]*p for _ in range(m)]\n",
            "    for i in range(m):\n",
            "        ai = A[i]\n",
            "        for k in range(n):\n",
            "            aik = ai[k]\n",
            "            if aik:\n",
            "                bj = B[k]\n",
            "                for j in range(p):\n",
            "                    result[i][j] += aik * bj[j]\n",
            "    return result\n",
            "def matmul(A, B):\n",
            "    ...\n",
            "def matmul(A, B):\n",
            "    ...\n"
          ]
        }
      ],
      "source": [
        "trainer.train()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tlaUdxC_VHpz"
      },
      "source": [
        "<a name=\"Inference\"></a>\n",
        "# Inference\n",
        "Now let's try the model we just trained!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8BZZHOKiF9Ct"
      },
      "outputs": [],
      "source": [
        "text = tokenizer.apply_chat_template(\n",
        "    [{\"role\": \"user\", \"content\": prompt}],\n",
        "    tokenize = False,\n",
        "    add_generation_prompt = True,\n",
        "    reasoning_effort = \"low\",\n",
        ")\n",
        "\n",
        "from transformers import TextStreamer\n",
        "_ = model.generate(\n",
        "    **tokenizer(text, return_tensors = \"pt\").to(\"cuda\"),\n",
        "    temperature = 1.0,\n",
        "    max_new_tokens = 1024,\n",
        "    streamer = TextStreamer(tokenizer, skip_prompt = False),\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-NUEmHFSYNTp"
      },
      "source": [
        "<a name=\"Save\"></a>\n",
        "### Saving to float16 or MXFP4 for VLLM\n",
        "\n",
        "We also support saving to `float16` directly. Select `merged_16bit` for float16 or `mxfp4` for MXFP4 (OpenAI's GPT-OSS native precision). We also allow `lora` adapters as a fallback. Use `push_to_hub_merged` to upload to your Hugging Face account! You can go to https://huggingface.co/settings/tokens for your personal tokens."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NjXGTkp7YNtB"
      },
      "outputs": [],
      "source": [
        "# Merge and push to hub in mxfp4 4bit format\n",
        "if False:\n",
        "    model.save_pretrained_merged(\"finetuned_model\", tokenizer, save_method = \"mxfp4\")\n",
        "if False: model.push_to_hub_merged(\"repo_id/repo_name\", tokenizer, token = \"hf...\", save_method = \"mxfp4\")\n",
        "\n",
        "# Merge and push to hub in 16bit\n",
        "if False:\n",
        "    model.save_pretrained_merged(\"finetuned_model\", tokenizer, save_method = \"merged_16bit\")\n",
        "if False: # Pushing to HF Hub\n",
        "    model.push_to_hub_merged(\"hf/gpt-oss-finetune\", tokenizer, save_method = \"merged_16bit\", token = \"\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V15Yhj1V9lwG"
      },
      "source": [
        "And we're done! If you have any questions on Unsloth, we have a [Discord](https://discord.gg/unsloth) channel! If you find any bugs or want to keep updated with the latest LLM stuff, or need help, join projects etc, feel free to join our Discord!\n",
        "\n",
        "Some other links:\n",
        "1. Train your own reasoning model - Llama GRPO notebook [Free Colab](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3.1_(8B)-GRPO.ipynb)\n",
        "2. Saving finetunes to Ollama. [Free notebook](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3_(8B)-Ollama.ipynb)\n",
        "3. Llama 3.2 Vision finetuning - Radiography use case. [Free Colab](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3.2_(11B)-Vision.ipynb)\n",
        "6. See notebooks for DPO, ORPO, Continued pretraining, conversational finetuning and more on our [documentation](https://docs.unsloth.ai/get-started/unsloth-notebooks)!\n",
        "\n",
        "<div class=\"align-center\">\n",
        "  <a href=\"https://unsloth.ai\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png\" width=\"115\"></a>\n",
        "  <a href=\"https://discord.gg/unsloth\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/Discord.png\" width=\"145\"></a>\n",
        "  <a href=\"https://docs.unsloth.ai/\"><img src=\"https://github.com/unslothai/unsloth/blob/main/images/documentation%20green%20button.png?raw=true\" width=\"125\"></a>\n",
        "\n",
        "  Join Discord if you need help + ⭐️ <i>Star us on <a href=\"https://github.com/unslothai/unsloth\">Github</a> </i> ⭐️\n",
        "</div>\n"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "T4",
      "include_colab_link": true,
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "011e3d0bb18541a5a2b0c071afb6eb32": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "023cb6271415454489dc5999bbba8a1b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "06fc35ae1cd245c2b711ae3d05004a35": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "07edfc41f3f44c9d851e12d984f19703": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0a49a476e7c844bd8048e4e9c056fa7b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_5b2ec551f1ca490886287c0230ae7e8f",
              "IPY_MODEL_9ddafad46ede4baebbc1f06928bbf01e",
              "IPY_MODEL_d893a89d9c764fbf98459a301ffaa06a"
            ],
            "layout": "IPY_MODEL_0fc9cecf55314af69389827852f390f3"
          }
        },
        "0c65a1aa0c7243d196e96bd6ef6ad556": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f2072d86f3824135a8c8d4fd0fe4c7f3",
            "max": 446,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_744f0a91396f4df6a3e5e4a32a785801",
            "value": 446
          }
        },
        "0f1c01032192466fa1aca285847f6391": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "0fc9cecf55314af69389827852f390f3": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "127fc2d8cfc54b978b2a49f8dc4c1d32": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "13d9d625e8734afdb9b1d896b2b0992e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a0e82f0030454bafb6bfb6bb1c9b3f7b",
            "placeholder": "​",
            "style": "IPY_MODEL_ea437a5e5ee741ac8ac528362dc45bb6",
            "value": " 1.19M/? [00:00&lt;00:00, 84.2MB/s]"
          }
        },
        "154717a8ed9745048a8906af3b221d41": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1947120737b344e996a45f935a18c2f0": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "194f8f1987c84e9992e6ae7ab8352874": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1b9a8cb9d8644923b2ee8b5b99de26da": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1cb6e34da8ad46b3a0057ab01de94fc1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e6cb50ddb26844a084ae5a7d4a492eca",
            "max": 1,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_515df8cf4bcf4fb0aa14705a195aced2",
            "value": 1
          }
        },
        "1f9374e2838647438922c6f3d16c2b6b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_011e3d0bb18541a5a2b0c071afb6eb32",
            "placeholder": "​",
            "style": "IPY_MODEL_0f1c01032192466fa1aca285847f6391",
            "value": " 4/4 [00:51&lt;00:00, 11.19s/it]"
          }
        },
        "229f715276c941479a98f632b6a4028e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b0963a510564455ba9182e877470d219",
            "placeholder": "​",
            "style": "IPY_MODEL_023cb6271415454489dc5999bbba8a1b",
            "value": " 3.37G/3.37G [01:34&lt;00:00, 84.5MB/s]"
          }
        },
        "2325e2f0d5d54bc685581d47c4045af2": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2619d2a3049a49c9ab567d5ace2e6da5": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "291b76fb815b419fb8d12001ffb53dcc": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_a9e779db1e7940e89b8599d4d19bf672",
              "IPY_MODEL_3f92e70ba76d4cc59309fe9dd9adbd31",
              "IPY_MODEL_4a6ea77ab6e547d0be2db28c3fc6a0de"
            ],
            "layout": "IPY_MODEL_9efc5450ab4241f7a4668af9d6af33e8"
          }
        },
        "2cff36ba4f974bd495bcc5808bc15f7b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3011422a32664f0b99d80456f17848b9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_46c0c957337744f19ee57dffa144bb88",
            "placeholder": "​",
            "style": "IPY_MODEL_479521faaed1410d9b18b6e136fdccae",
            "value": " 1.16G/1.16G [00:12&lt;00:00, 77.9MB/s]"
          }
        },
        "3314a4aba3e44ab59c7265d2fbb8f101": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "380159861a234f5aa9a0737bd7f03bcb": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "39dd71d4a7d741dba0d95c6964207bf8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2cff36ba4f974bd495bcc5808bc15f7b",
            "placeholder": "​",
            "style": "IPY_MODEL_84dd3e02fd8249189c93a017f7e1cc93",
            "value": "generation_config.json: 100%"
          }
        },
        "39fc8be5ba8141d09eed02162d2512bf": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3bd095f10eb24396aeec9d84629190d3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3e89a1c5e2ed49b280947c51e1713483": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3f92e70ba76d4cc59309fe9dd9adbd31": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8b68fbede6a04c95a8996bf9e7945a68",
            "max": 1,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_1947120737b344e996a45f935a18c2f0",
            "value": 1
          }
        },
        "404ff34787884e879ab8e8c1cb99250f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "42c96fe6e04743b994950ce8f774b645": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "451add29f7d84acfad95b1fa96a5cdd2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_bcedab6c5dd0423b91eea9734e5b6904",
            "placeholder": "​",
            "style": "IPY_MODEL_528d3a190af640d9ab7a7d7fe086f31c",
            "value": " 165/165 [00:00&lt;00:00, 18.6kB/s]"
          }
        },
        "46c0c957337744f19ee57dffa144bb88": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "479521faaed1410d9b18b6e136fdccae": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "49e3ddd3f92146baa5a8cb017a036252": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4a6ea77ab6e547d0be2db28c3fc6a0de": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3e89a1c5e2ed49b280947c51e1713483",
            "placeholder": "​",
            "style": "IPY_MODEL_7b808206dd32488c8d62888934c3ee12",
            "value": " 15.1k/? [00:00&lt;00:00, 888kB/s]"
          }
        },
        "508965f1eba9451f8d07cc5ce4cd1e09": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9a76637b3afd4c56be2d99c200c5de8e",
            "max": 4,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_e397e262339c437593d7159870e4ddd9",
            "value": 4
          }
        },
        "515df8cf4bcf4fb0aa14705a195aced2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "528d3a190af640d9ab7a7d7fe086f31c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "5323ebbabec14d7dad77fc47538eba93": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "57112275643b4fce8f681ac9d21bc2b1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "59259ca023ec4f739f723dccb312bca3": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "59fa8c137a1649559e4d3c7486cbdf30": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5b2ec551f1ca490886287c0230ae7e8f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_7c2a5bfc484043fe8c83d766c84b1827",
            "placeholder": "​",
            "style": "IPY_MODEL_d481cf7937ea4399adf8153a672db38d",
            "value": "tokenizer.json: 100%"
          }
        },
        "5cae2f7900ca4509a68d345e26c738e5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_bbc57ccb80df4cb68c271c3ffcb85bce",
              "IPY_MODEL_9809a8d0fa1d4bf8b5a96996ad44e9c5",
              "IPY_MODEL_a2ee831695754546845b552ab4c880ad"
            ],
            "layout": "IPY_MODEL_59259ca023ec4f739f723dccb312bca3"
          }
        },
        "6126ce31d40c4afbbc4f30ff618c5105": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_6162fa10939c49feb3cbb5a25bd31f70",
              "IPY_MODEL_508965f1eba9451f8d07cc5ce4cd1e09",
              "IPY_MODEL_1f9374e2838647438922c6f3d16c2b6b"
            ],
            "layout": "IPY_MODEL_2619d2a3049a49c9ab567d5ace2e6da5"
          }
        },
        "6162fa10939c49feb3cbb5a25bd31f70": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d5841e9d82a44eabb3a10c6990770b26",
            "placeholder": "​",
            "style": "IPY_MODEL_c9e25b142c284eb585d7fef89e9cfb20",
            "value": "Loading checkpoint shards: 100%"
          }
        },
        "667c64b9f6294f5a85e6653c220d3c19": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2325e2f0d5d54bc685581d47c4045af2",
            "max": 1158267008,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_42c96fe6e04743b994950ce8f774b645",
            "value": 1158267008
          }
        },
        "6b2ef4d303ee437cb19a2ad0800e3d9e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_81aa522133724d51adc0e1f19cc0bf8f",
              "IPY_MODEL_8c2b98be4b2a466d834d2c8b918946f7",
              "IPY_MODEL_e4275bea01d24aad92bf3b52c0f6032f"
            ],
            "layout": "IPY_MODEL_3314a4aba3e44ab59c7265d2fbb8f101"
          }
        },
        "6dda7866d8c14e88be4fbd797417f556": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6f3bfc7e482147999b1fcf399e669452": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6f5b6b3036c34a248657e83d2db979ff": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "719839a5728a46248edcda02252e9561": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_ddfbeb16103f44a08e3d7bb6bf4da86a",
              "IPY_MODEL_1cb6e34da8ad46b3a0057ab01de94fc1",
              "IPY_MODEL_ff698edf61144077bf61fe03369cdf5b"
            ],
            "layout": "IPY_MODEL_07edfc41f3f44c9d851e12d984f19703"
          }
        },
        "72469601dc8f485dbe1065124f86ad0b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "7321c679967f4e20b1feb305cb9d5e46": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "744f0a91396f4df6a3e5e4a32a785801": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "79c650567e584ddc9834ef06ef8e7309": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "7b808206dd32488c8d62888934c3ee12": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "7c2a5bfc484043fe8c83d766c84b1827": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7dceab848c9043bfbad62c0ac6d9f13c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_127fc2d8cfc54b978b2a49f8dc4c1d32",
            "placeholder": "​",
            "style": "IPY_MODEL_49e3ddd3f92146baa5a8cb017a036252",
            "value": "Fetching 4 files: 100%"
          }
        },
        "7e7be5dbde274fb1bd2b673fa17e3c6c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "805f7c86609347ed8fe945a38111edf3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "81aa522133724d51adc0e1f19cc0bf8f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c7551368a9e74325986547cf00feb51f",
            "placeholder": "​",
            "style": "IPY_MODEL_6f5b6b3036c34a248657e83d2db979ff",
            "value": "model-00001-of-00004.safetensors: 100%"
          }
        },
        "84dd3e02fd8249189c93a017f7e1cc93": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "856ded5e0d6e491eb38d316f2f43fbd7": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": "20px"
          }
        },
        "85dac41978864ba78784413758a2e393": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_194f8f1987c84e9992e6ae7ab8352874",
            "placeholder": "​",
            "style": "IPY_MODEL_3bd095f10eb24396aeec9d84629190d3",
            "value": "special_tokens_map.json: 100%"
          }
        },
        "882a51f080ed407ebe01a05b970e48a5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_aa9017bafbe84475a843d4116e2d8cc1",
              "IPY_MODEL_9b62f5baa62f4bccb8c2b7da62c280c7",
              "IPY_MODEL_229f715276c941479a98f632b6a4028e"
            ],
            "layout": "IPY_MODEL_99b4fa0ecb204d41a4d9237b604fb8e5"
          }
        },
        "89ae701ad8ae4531bccbfaf2df88ee0f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_fe4ac2a3569c44c5813b447609931a2e",
            "placeholder": "​",
            "style": "IPY_MODEL_79c650567e584ddc9834ef06ef8e7309",
            "value": "model-00004-of-00004.safetensors: 100%"
          }
        },
        "8b68fbede6a04c95a8996bf9e7945a68": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": "20px"
          }
        },
        "8c2b98be4b2a466d834d2c8b918946f7": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d4a2d9865d6641b4891de7d61d465f12",
            "max": 3998751275,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_e299af20255f4b96bb66c649e12dafc8",
            "value": 3998751275
          }
        },
        "8d2807bfe85447bdad67987460e99168": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "8f439204fa79466aa2d20d0d2a28d74d": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9190b3de6fcd4e0ca50c20c055eef834": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_dea001037d0b479585264f675754415a",
            "placeholder": "​",
            "style": "IPY_MODEL_e913fb6b7b86498d847c64196478a628",
            "value": " 4/4 [01:38&lt;00:00, 98.76s/it]"
          }
        },
        "925b4741bcf24e52a91070b1c2a133d5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "95461ebf2f9d44c9b745da5c35d3ecea": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9809a8d0fa1d4bf8b5a96996ad44e9c5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_39fc8be5ba8141d09eed02162d2512bf",
            "max": 3996690997,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_380159861a234f5aa9a0737bd7f03bcb",
            "value": 3996690997
          }
        },
        "99b4fa0ecb204d41a4d9237b604fb8e5": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "99e4796e59e0472eac83077cff768bf0": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6dda7866d8c14e88be4fbd797417f556",
            "placeholder": "​",
            "style": "IPY_MODEL_c535608487724a408d60b5d1c54b18b8",
            "value": "model.safetensors.index.json: "
          }
        },
        "9a76637b3afd4c56be2d99c200c5de8e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9b62f5baa62f4bccb8c2b7da62c280c7": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_95461ebf2f9d44c9b745da5c35d3ecea",
            "max": 3372033380,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_a7be22b2a2cb4d7ebfa5016cd081af86",
            "value": 3372033380
          }
        },
        "9ddafad46ede4baebbc1f06928bbf01e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c66e476d9fb046499f5dc878844b9aa8",
            "max": 27868174,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_72469601dc8f485dbe1065124f86ad0b",
            "value": 27868174
          }
        },
        "9efc5450ab4241f7a4668af9d6af33e8": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9fe343b36ef1493b86fc6a788bd916ed": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a0e82f0030454bafb6bfb6bb1c9b3f7b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a26fca95ca7f484fa3fbd89bf3736fa1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ab126a046cdd44e0a047abcb1528113d",
            "max": 165,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_06fc35ae1cd245c2b711ae3d05004a35",
            "value": 165
          }
        },
        "a2ee831695754546845b552ab4c880ad": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5323ebbabec14d7dad77fc47538eba93",
            "placeholder": "​",
            "style": "IPY_MODEL_6f3bfc7e482147999b1fcf399e669452",
            "value": " 4.00G/4.00G [01:38&lt;00:00, 99.8MB/s]"
          }
        },
        "a461767972644fdda27f372c6c0d983a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a4d88610a9464bbf9be3922d760b0670": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a715051a1252454880f83726c380857f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a7b21195b2824db89f19f8695fcb56cd": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a7be22b2a2cb4d7ebfa5016cd081af86": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "a9e779db1e7940e89b8599d4d19bf672": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c103676f50c549d981e719d8baece21a",
            "placeholder": "​",
            "style": "IPY_MODEL_57112275643b4fce8f681ac9d21bc2b1",
            "value": "chat_template.jinja: "
          }
        },
        "aa9017bafbe84475a843d4116e2d8cc1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_59fa8c137a1649559e4d3c7486cbdf30",
            "placeholder": "​",
            "style": "IPY_MODEL_c39b4f53a4fb409fb414b26a5f776eb8",
            "value": "model-00003-of-00004.safetensors: 100%"
          }
        },
        "ab126a046cdd44e0a047abcb1528113d": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "adfde043c5694bc381fef6d0253ee113": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_89ae701ad8ae4531bccbfaf2df88ee0f",
              "IPY_MODEL_667c64b9f6294f5a85e6653c220d3c19",
              "IPY_MODEL_3011422a32664f0b99d80456f17848b9"
            ],
            "layout": "IPY_MODEL_7e7be5dbde274fb1bd2b673fa17e3c6c"
          }
        },
        "b0963a510564455ba9182e877470d219": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bb788875516e4aba9ec87406be52eeff": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_856ded5e0d6e491eb38d316f2f43fbd7",
            "max": 1,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_d09aa36b14ff466d96b1236be27919de",
            "value": 1
          }
        },
        "bbc57ccb80df4cb68c271c3ffcb85bce": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_df7811d1db294affbb1f7400b6d67393",
            "placeholder": "​",
            "style": "IPY_MODEL_bfde109172ab414092bd3c0618e6ac77",
            "value": "model-00002-of-00004.safetensors: 100%"
          }
        },
        "bcedab6c5dd0423b91eea9734e5b6904": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bfde109172ab414092bd3c0618e6ac77": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c103676f50c549d981e719d8baece21a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c22e6f19fcd949738d3da38670209bf1": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c39b4f53a4fb409fb414b26a5f776eb8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c535608487724a408d60b5d1c54b18b8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c66e476d9fb046499f5dc878844b9aa8": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c7551368a9e74325986547cf00feb51f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c7c83df93e1444baae6168970d498ba9": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c9e25b142c284eb585d7fef89e9cfb20": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "ce40e56d7a524059a366371d52a5d7f5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_99e4796e59e0472eac83077cff768bf0",
              "IPY_MODEL_bb788875516e4aba9ec87406be52eeff",
              "IPY_MODEL_13d9d625e8734afdb9b1d896b2b0992e"
            ],
            "layout": "IPY_MODEL_154717a8ed9745048a8906af3b221d41"
          }
        },
        "d09aa36b14ff466d96b1236be27919de": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "d481cf7937ea4399adf8153a672db38d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d4a2d9865d6641b4891de7d61d465f12": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d5841e9d82a44eabb3a10c6990770b26": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d893a89d9c764fbf98459a301ffaa06a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_404ff34787884e879ab8e8c1cb99250f",
            "placeholder": "​",
            "style": "IPY_MODEL_9fe343b36ef1493b86fc6a788bd916ed",
            "value": " 27.9M/27.9M [00:01&lt;00:00, 22.0MB/s]"
          }
        },
        "db16c5bfe0be4728a4cacda8573101c4": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "db814fd85f5342539537014ff47a4679": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_85dac41978864ba78784413758a2e393",
              "IPY_MODEL_0c65a1aa0c7243d196e96bd6ef6ad556",
              "IPY_MODEL_e19c28cd15104cff8bff0bc3a1338f3f"
            ],
            "layout": "IPY_MODEL_c22e6f19fcd949738d3da38670209bf1"
          }
        },
        "ddfbeb16103f44a08e3d7bb6bf4da86a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8f439204fa79466aa2d20d0d2a28d74d",
            "placeholder": "​",
            "style": "IPY_MODEL_805f7c86609347ed8fe945a38111edf3",
            "value": "tokenizer_config.json: "
          }
        },
        "dea001037d0b479585264f675754415a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "df7811d1db294affbb1f7400b6d67393": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e19c28cd15104cff8bff0bc3a1338f3f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a715051a1252454880f83726c380857f",
            "placeholder": "​",
            "style": "IPY_MODEL_925b4741bcf24e52a91070b1c2a133d5",
            "value": " 446/446 [00:00&lt;00:00, 33.3kB/s]"
          }
        },
        "e25e6d26258a4d5e864a45b7a3107c4f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c7c83df93e1444baae6168970d498ba9",
            "max": 4,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_8d2807bfe85447bdad67987460e99168",
            "value": 4
          }
        },
        "e299af20255f4b96bb66c649e12dafc8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "e397e262339c437593d7159870e4ddd9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "e4275bea01d24aad92bf3b52c0f6032f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1b9a8cb9d8644923b2ee8b5b99de26da",
            "placeholder": "​",
            "style": "IPY_MODEL_7321c679967f4e20b1feb305cb9d5e46",
            "value": " 4.00G/4.00G [01:38&lt;00:00, 25.4MB/s]"
          }
        },
        "e6cb50ddb26844a084ae5a7d4a492eca": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": "20px"
          }
        },
        "e913fb6b7b86498d847c64196478a628": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "ea437a5e5ee741ac8ac528362dc45bb6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f0284fe69116476685139bc8ad7d80d2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_7dceab848c9043bfbad62c0ac6d9f13c",
              "IPY_MODEL_e25e6d26258a4d5e864a45b7a3107c4f",
              "IPY_MODEL_9190b3de6fcd4e0ca50c20c055eef834"
            ],
            "layout": "IPY_MODEL_a7b21195b2824db89f19f8695fcb56cd"
          }
        },
        "f2072d86f3824135a8c8d4fd0fe4c7f3": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "fe4ac2a3569c44c5813b447609931a2e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ff63b135c33544e6b8f327817b04e183": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_39dd71d4a7d741dba0d95c6964207bf8",
              "IPY_MODEL_a26fca95ca7f484fa3fbd89bf3736fa1",
              "IPY_MODEL_451add29f7d84acfad95b1fa96a5cdd2"
            ],
            "layout": "IPY_MODEL_a461767972644fdda27f372c6c0d983a"
          }
        },
        "ff698edf61144077bf61fe03369cdf5b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_db16c5bfe0be4728a4cacda8573101c4",
            "placeholder": "​",
            "style": "IPY_MODEL_a4d88610a9464bbf9be3922d760b0670",
            "value": " 22.8k/? [00:00&lt;00:00, 1.64MB/s]"
          }
        },
        "state": {}
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
